Computer simulations of language change notes
This website collects my personal notes on Computer simulations of language change. These notes are provided to bring full transparency to my research process. Of course, since they are only notes, they do not reflect my final thoughts on a topic, and should not be interpreted as such. To read finished papers, please consult my website. Do not use these notes as a basis for your own scientific research. Start from high-quality, peer-reviewed scientific literature instead.
Steven Pinker
language evolution, a recent field
Consensus
1. interdisciplinarity
2. growing interest in mathematical and computational modelling
Models are useful because they allow researchers to test particular theories about the mechanisms underlying the evolution of language. Given the number of different factors that may potentially influence language evolution, our intuitions about their complex interactions are often limited. It is exactly in these circumstances, when multiple processes have to be considered together, that modelling becomes a useful – and perhaps even necessary – tool.
In this book, modelling work has been used to inform theories about biological adaptations for grammar (Pinker, Dunbar, Briscoe, Komarova and Nowak, Chapters 2, 12, 16, and 17), about the emergence of language structure through cultural transmission (Hurford, Deacon, Kirby and Christiansen, Chapters 3, 7, and 15), and about the evolution of phonetic gesture systems (Studdert-Kennedy and Goldsten, Chapter 13).
We envisage that the interest in mathematical and computational modelling is likely to increase even further, especially as it becomes more sophisticated in terms of both psychological mechanisms and linguistic complexity.
3. pre-adaptations necessary
Disagreement
1. constraints on language structure
… no further controversies explicitly mentioned
Language itself is rather difficult to define, existing as it does both as transitory utterances that leave no trace, and as patterns of neural connectivity in the natural world’s most complex brains. It is never stationary, changing over time and within populations which themselves are dynamic. It is infinitely flexible and (almost) universally present. It is by far the most complex behaviour we know of […].
central theory
mental lexicon
word
bidirectionality of symbols
grammar
parts of the mind interfacing with language
nvda. Cognitive Grammar assumes many of these aspects are NOT separated and part of the same structure.
adaptation
language as general cognitive abilities
(arguments for a language faculty)
theory
the cognitive niche
1. deception
(Dawkins and Krebs)
2. thinking
3. grooming
4. courtship device
Evolutionary game theory has allowed biologists to predict how organisms ought to interact with other organisms co-evolving their own strategies (Maynard Smith 1982). Language, like sex, aggression and cooperation, is a game it takes two to play, and game theory can provide the external criteria for utility enjoyed by the rest of evolutionary biology.
Modellers assume only that the transmission of information between partners provides them with an advantage (say, by exchanging information or coordinating their behaviour), and that the advantage translates into more offspring, with similar communicative skills.
The question then is how a stable communication system might evolve from repeated pairwise interactions and, crucially, whether such systems have the major design features of human language.
1. Hurford (1989)
2. Nowak et al. (1999, 2000)
Mathematical models and computer simulations can show that the advantages claimed for some features of language really can evolve by known mechanisms of natural selection. These models cannot, of course, show that language in fact evolved according to the proposed scenario.
(gene things I don’t understand)
James R. Hurford
key to explaining complex phenomena of homan language
| biological evolution | historical evolution |
|---|---|
| of the language capacity | of individual languages |
pre-adaptation
For example, bipedalism set in train anatomical changes which culminated in the human vocal tract. Though speech is clearly adaptive, bipedalism itself it not an adaptation for speech; it is a pre-adaptation.
↓
(this list can be used as a checklist for the computer models to highlight was and wasn’t been modelled)
| historical linguistics | language evolution |
|---|---|
| uniformitarianism | simple to complex |
| same principles then as now | complex principles have arisen out of simple principles |
grammaticalisation
This is still a very formal way of looking at things. Still, this was 2003, so cut the man a little slack.
unidirectionality
computer modellers of emerging language
A survey of some of these works, analysing their principle dimensions, and the issues they raise, appears in Hurford (2002).
iterated learning models
This strand of computational simulation research has the potential to clarify the essentials of the interaction between (a) the psychological capacities of language learners and (b) the historical dynamics of populations of learners giving rise to complex grammars resembling the grammars of real natural languages.
In such simulations, a population of agents begins with no shared system of communication. The agents are ‘innately’ endowed with certain competencies, typically including control of a space of possible meanings, an inventory of possible signals, and a capacity for acquiring grammars of certain specified sorts on exposure to examples of meaning-signal pairs.
The simulations typically proceed with each generation learning from its predecessor, on the basis of observation of its communicative behaviour. At first, there is no coherent communicative behaviour in the simulated population. Over time, a coherent shared syntactic system emerges. The syntactic systems which have been achieved in this research paradigm are all, of course, simpler than real attested languages, but nevertheless possess many of the central traits of natural language syntactic organisation, including recursivity, compositionality of meaning, asymmetric distribution of regular and irregular forms according to frequency, grammatical functional elements with no denotational meaning, grammatical markers of passive voice and of reflexivity, and elementary partitioning into phrases.
computer simulations and invisible hand
Simulations within an ILM framework strip the interaction between individuals down to the bare minimum from which language-like systems can be shown to emerge. The key property of these models is that each new generation learns its language from a restricted set of exemplars produced by the preceding generation.
Robin I.M. Dunbar
social hypothesis
not relevant to me
linguistic diversification
↓
1. drift
2. need to differentiate communities
Beyond this, however, we have little real understanding of the processes involved in either dialect change or language evolution, or for that matter in the functions that these processes subserve. We assume that these functions are largely social, and we have some understanding of the types of process that can precipitate language change (trade, colonisation, emulation of culturally or economically superior groups, etc.), but by and large there is little other than conjecture to explain why these processes exist or why they should work in the way they do.
Part of the problem is, of course, the temporal scale on which these changes occur (generations in the case of dialects, perhaps lmillennia in the case of languages). Inevitably, this makes it all but impossible for us to observe the process first-hand.
Simon Kirby and Morten H. Christiansen
| animal communication | human communication |
|---|---|
| mostly determined by genes | learnt to a very high degree |
↓ why?
size of human language
↳ language learning
The environment that the learner interacts with consists of the ‘output’ of that learning by other individuals.
Universal features of language (which constrain it)
1. digital infinity
2. compositionality
3. typological properties
object of study
How do we explain universals that appear to be unique to language?
SRN
sequence-based learning
learnability score
↳ branching consistency
This is clearly a striking result. Why postulate a domain-specific constraint if the data that constraint should account for is predicted by a general model of sequential learning?
This type of result was also found in an experimental setting (Christiansen and Ellefson 2002).
What these experiments with ALL [(artificial language learning]) and SRNs show us is that we should be careful about ascribing universal properties of language to a domain-specific innate bias. We argue that an explanation that appeals to non-linguistic biases should be preferred where possible. Simplistic, all-or-nothing explanations should be avoided, however.
Language learning. The child’s learning mechanisms take linguistic data (utterances) and generate some kind of internal representation of a language. The range of possible languages is determined by the structure of the learning mechanisms (i.e. its prior biases).
↓
where do the data to learn from come from?
feedback loop
Iterated learning. The input to learning is the product of the acquired language of others. There are now two dynamical systems that contribute to the range of possible languages: individual learning and social/cultural transmission.
Iterated Learning Model (ILM)
evolutionary model
compositionality
computer modelling
Typical components of simulation models
1. a population of agents
2. a space of possible signals
3. a space of possible meanings
4. a production model
5. a learning model
The fact that the learners are given meanings as well as signals seems unrealistic. Ultimately, simulations of the process of iterated learning will need to enrich the model with contexts. Whereas meanings are private and inaccessible to learners, contexts are public and may allow the inference of meanings. See e.g. Steels et al. (200) and Smith (2001) for discussion of these fascinating extensions to the model.
↳ common property of ILM simulations
Composition of the simulation
initialisation of a simulation
poverty of the stimulus
stabilisation
ideal distribution
Explaining the behaviour of simulations
How are these languages stable, and why are they compositional?
The initial holistic languages are unstable by virtue of the poverty of the stimulus, which acts as a bottleneck on the transmission of language. In the early stages of the simulation, the population is essentially randomly searching around the space of possible meaning-string pairs, driven by the learner’s failure to generalise (non-randomly) to unheard meanings.
At some point a learner will infer some form of non-random behaviour in a speaker and use this to generalize to other meanings. In the first instance, this inference if ‘rule-like’ behaviour will actually be ill-founded (since the speaker will have been behaving randomly). Nevertheless, this learner will now produce utterances that are, to a small extent, non-random.
chain of generalisation
end result
Why do some kinds of holism persist?
frequency effects
↳ bottleneck explanation
If, for reasons such as language contact of processes of phonological erosion, irregulars make their way into a language, the pressure to regularise them will be strongest in the low-frequency parts of the system.
An interesting by-product of the introduction of frequency biases to the meaning space is the removal of a fixed endpoint to the simulations. The language in this model is always changing – but not so much that speaker-to-speaker intelligibility is degraded. This is another way in which these simulation results seem to mirror what we know about language more accurately.
| horizontal transmission | vertical transmission |
|---|---|
| learners mainly learn from adults | a lot of contact between learners |
| language changes slowly | language changes fast |
| may take many generations to stabilise on a structured system | structure can emerge rapidly |
Three principal assumptions of IL models
By systematically varying the representations of meanings to which the agents in the ILM have access, we are able to see under which circumstances structured mappings between meanings and signals will emerge.
biological evolution
Evolutionary iterated learning. The cognitive mechanisms (and therefore prior biases) of an individual language learner are provided through genetic transmission, which necessarily involves selection. The structure of language arises from the interaction of three systems of information transmission.
Computer simulations of language change notes
This website collects my personal notes on Computer simulations of language change. These notes are provided to bring full transparency to my research process. Of course, since they are only notes, they do not reflect my final thoughts on a topic, and should not be interpreted as such. To read finished papers, please consult my website. Do not use these notes as a basis for your own scientific research. Start from high-quality, peer-reviewed scientific literature instead.